Distribution of tasks among asymmetric processing elements

ABSTRACT

A technique to promote determinism among multiple clocking domains within a computer system or integrated circuit. In one embodiment, one or more execution units are placed in a deterministic state with respect to multiple clocks within a processor system having a number of different clocking domains.

FIELD OF THE INVENTION

Embodiments of the invention relate generally to the field of information processing and more specifically, to the field of distributing program tasks among various processing elements.

BACKGROUND

As more processing throughput is required from modern microprocessors, it is often at the expense of power consumption. Some applications, such as mobile internet devices (MIDs), ultra-mobile personal computers (UMPCs), cellular phones, personal digital assistants (PDAs), and even laptop/notebook computers, may benefit from processors that consume relatively little power. However, achieving relatively high processing throughput at relatively low power is a challenge, involving various design trade-offs, depending on the usage models of the computing platform.

One approach to reducing power in a computing platform when there is relatively little activity, is to place the processor in a low-power state. However, placing a processor in a low-power state or returning a processor from a low-power state may require a non-trivial amount of time. Therefore, it may or may not be worth the time required to place a processor in a low-power state or to return the processor from a low-power state. Furthermore, not all processes and tasks that are run on a processor require the full processing throughput of the processor.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:

FIG. 1 illustrates a block diagram of a microprocessor, in which at least one embodiment of the invention may be used;

FIG. 2 illustrates a block diagram of a shared bus computer system, in which at least one embodiment of the invention may be used;

FIG. 3 illustrates a block diagram a point-to-point interconnect computer system, in which at least one embodiment of the invention may be used;

FIG. 4 is a curve showing the relationship between power and performance using at least one embodiment of the invention;

FIG. 5 is a flow diagram of operations that may be used for performing at least one embodiment of the invention;

FIG. 6 illustrates a number of processing units and an activity level, thermal, or power detection/monitoring unit that may be used in at least one embodiment.

DETAILED DESCRIPTION

Embodiments of the invention include a microprocessor or processing system having a number of asymmetric processing elements. In some embodiments, each processing element is a processor core, having one or more execution resources, such as arithmetic logic units (ALUs), instruction decoder, and instruction retirement unit, among other things. In some embodiments, the number of asymmetric processing elements has at least two different processing throughput or performance capabilities, power consumption characteristics or limits, voltage supply requirements, clock frequency characteristics, number of transistors, and/or instruction set architectures (ISAs). In one embodiment, an asymmetric microprocessor includes at least one main processor core having larger power consumption characteristics and/or processing throughput/performance characteristics than at least one other processing core within or otherwise associated with the microprocessor.

In one embodiment, a process or task running or intended to run on a main higher power/performance processing core may be transferred to one of the other lower power/performance processing cores for various reasons, including that the process or task does not require the processing throughput of one of the main cores, the processor or the system in which it's used is placed into or otherwise requires a lower-power consumption condition (such as when running on battery power), and for increasing the processing throughput of the asymmetric microprocessor or system in which the higher power/performance cores and lower power/performance cores are used. For example, in one embodiment, the asymmetric processing elements may be used concurrently or otherwise in parallel to perform multiple tasks or processes, thereby improving the overall throughput of the processor and processing system.

In one embodiment, the at least one main processing core has a different ISA than at least one of the at least one processor cores having a lower power consumption characteristic and/or processing performance capability. In one embodiment, instruction translation logic in the form of hardware, software, or some combination thereof, may be used to translate instructions for the at least one main processor core into instructions for the at least one other lower-power/performance processing core. For example, in one embodiment, one or more of the main higher power/performance cores may have a complex instruction set computing (CISC) architecture, such as the “x86” computing architecture, and therefore performs instructions that are intended for x86 processor cores. One or more of the lower power/performance cores may have a different ISA than the main core, including a reduced instruction set computing (RISC) architecture, such as an Advanced RISC Machine (ARM) core. In other embodiments, the main processing element(s) and the lower power/performance processing element(s) may include other architectures, such as the MIPS ISA. In other embodiments the main processing element(s) may have the same ISA as the lower power/performance element(s) (e.g., x86).

In one embodiment, a number of different threads, processes, or tasks associated with one or more software programs may be intelligently moved among and ran on a number of different processing elements, having a number of different processing capabilities (e.g., operating voltage, performance, power consumption, clock frequency, pipeline depth, transistor leakage, ISA), according to the dynamic performance and power consumption needs of the processor or computer system. For example, if one process, such as that associated with a spreadsheet application, does not require the full processing capabilities of a main, higher performance processor core, but may be instead be ran with acceptable performance on a lower-power core, the process may be transferred to or otherwise ran on the lower power core and the main, higher power processor core may be placed in a low power state or may just remain idle. By running threads/processes/tasks on a processor core that better matches the performance needs of the thread/process/task, power consumption may be optimized, according to some embodiments.

FIG. 1 illustrates a microprocessor in which at least one embodiment of the invention may be used. In particular, FIG. 1 illustrates microprocessor 100 having one or more main processor cores 105 and 110, each being able to operate at a higher performance level (e.g., instruction throughput) or otherwise consume more power than one or more low-power cores 115, 120. In one embodiment, the low-power cores may be operated at the same or different operating voltage as the main cores. Furthermore, in some embodiments, the low-power cores may operate a different clock speed or have fewer execution resources, such that they operate at a lower performance level than the main cores.

In other embodiments, the low-power cores may be of a different ISA than the main cores. For example, the low-power cores may have an ARM ISA and the main cores may have an x86 ISA, such that a program using x86 instructions may need to have these instructions translated into ARM instructions if a process/task/thread is transferred to one of the ARM cores. Because the process/thread/task being transferred may be one that does not require the performance of one of the main cores, a certain amount of latency associated with the instruction translation may be tolerated without noticeable or significant loss of performance.

Also illustrated in FIG. 1 is at least one other non-CPU functional unit 117, 118, and 119 which may perform other non-CPU related operations. In one embodiment, the functional units 117, 118, and 119 may include functions such as graphics processing, memory control and I/O or peripheral control, such as audio, video, disk control, digital signal processing, etc. The multi-core processor of FIG. 1 also illustrates a cache 123 that each core can access for data or instructions corresponding to any of the cores.

In one embodiment, logic 129 may be used to monitor performance or power of any of the cores illustrated in FIG. 1 in order to determine whether a process/task/thread should be migrated from one core to another to optimize power and performance. In one embodiment, logic 129 is associated with the main cores 105 and 110 to monitor an activity level of the cores to determine whether the processes/threads/tasks running on those cores could be run on a lower-power core 115, 120 at an acceptable performance level, thereby reducing the overall power consumption of the processor. In other embodiments, logic 129 may respond to a power state of the system, such as when the system goes from being plugged into an A/C outlet to battery power. In this case, the OS or some other power state monitoring logic may inform logic 129 of the new power conditions and the logic 129 may cause a current-running process (or processes yet to be scheduled to run) to either be transferred (or scheduled) to a lower-power core (in the case of going from A/C to battery, for example) or from a lower-power core to a main core (in the case of going from battery to A/C, for example). In some embodiments, an operating system (OS) may be responsible for monitoring or otherwise controlling the power states of the processor and/or system, such that the logic 129 simply reacts to the OS's commands to reduce power by migrating tasks/threads/processes to a core that better matches the performance needs of the tasks/threads/processes while accomplishing the power requirements dictated or indicated by the OS.

In some embodiments, the logic 129 may be hardware logic or software, which may or may not determine a core(s) on which a process/task/thread should be run independently of the OS. In one embodiment, for example, logic 129 is implemented in software to monitor the activity level of the cores, such as the main cores, to see if it drops below a threshold level, and in response thereto, causes one or more processes running on the monitored core(s) to be transferred to a lower-power core, such as cores 115 and 120. Conversely, logic 129 may monitor the activity level of a process running on a lower-power core 115 and 120 in order to determine whether it is rising above a threshold level, thereby indicating the process should be transferred to one of the main cores 105, 110. In other embodiments, logic 129 may independently monitor other performance or power indicators within the processor or system and cause processes/threads/tasks to be migrated to cores that more closely fit the performance needs of the tasks/processes/threads while meeting the power requirements of the processor of the system at a given time. In this way, the power and performance of processor 100 can be controlled without the programmer or OS being concerned or even aware of the underlying power state of the processor.

In other embodiments, each core in FIG. 1 may be concurrently running different tasks/threads/processes to get the most performance benefit possible from the processor. For example, in one embodiment, a process/thread/task that requires high performance may be run on a main core 105, 110 concurrently with a process/thread/task that doesn't require as high performance as what the main cores are able to deliver on lower-power cores 115, 120. In one embodiment, the programmer determines where to schedule these tasks/threads/processes, whereas in other embodiments, these threads/tasks/processes may be scheduled by an intelligent thread scheduler (not shown) that is aware of the performance capabilities of each core and can schedule the threads to the appropriate core accordingly. In other embodiments, the threads are simply scheduled without regard to the performance capabilities of the underlying cores and the threads/processes/tasks are migrated to a more appropriate core after the activity levels of the cores in response to the threads/processes/tasks are determined. In this manner, neither an OS nor a programmer need be concerned about where the threads/processes/tasks are scheduled, because the threads/processes/tasks are scheduled on the appropriate core(s) that best suits the performance requirement of each thread while maintaining the power requirements of the system or processor.

In one embodiment, logic 129 may be hardware, software, or some combination thereof. Furthermore, logic 129 may be distributed within one or more cores or exist outside the cores while maintaining electronic connection to the one or more cores to monitor activity/power and cause threads/tasks/processes to be transferred to appropriate cores.

FIG. 2, for example, illustrates a front-side-bus (FSB) computer system in which one embodiment of the invention may be used. Any processor 201, 205, 210, or 215 may include asymmetric cores (differing in performance, power, operating voltage, clock speed, or ISA), which may access information from any local level one (L1) cache memory 220, 225, 230, 235, 240, 245, 250, 255 within or otherwise associated with one of the processor cores 223, 227, 233, 237, 243, 247, 253, 257. Furthermore, any processor 201, 205, 210, or 215 may access information from any one of the shared level two (L2) caches 203, 207, 213, 217 or from system memory 260 via chipset 265. One or more of the processors in FIG. 2 may include or otherwise be associated with logic 219 to monitor and/or control the scheduling or migration of processes/threads/tasks between each of the asymmetric cores of each processor. In one embodiment, logic 219 may be used to schedule or migrate threads/tasks/processes to or from one asymmetric core in one processor to another asymmetric core in another processor.

In addition to the FSB computer system illustrated in FIG. 2, other system configurations may be used in conjunction with various embodiments of the invention, including point-to-point (P2P) interconnect systems and ring interconnect systems. The P2P system of FIG. 3, for example, may include several processors, of which only two, processors 370, 380 are shown by example. Processors 370, 380 may each include a local memory controller hub (MCH) 372, 382 to connect with memory 32, 34. Processors 370, 380 may exchange data via a point-to-point (PtP) interface 350 using PtP interface circuits 378, 388. Processors 370, 380 may each exchange data with a chipset 390 via individual PtP interfaces 352, 354 using point to point interface circuits 376, 394, 386, 398. Chipset 390 may also exchange data with a high-performance graphics circuit 338 via a high-performance graphics interface 339. Embodiments of the invention may be located within any processor having any number of processing cores, or within each of the PtP bus agents of FIG. 3. In one embodiment, any processor core may include or otherwise be associated with a local cache memory (not shown). Furthermore, a shared cache (not shown) may be included in either processor outside of both processors, yet connected with the processors via p2p interconnect, such that either or both processors' local cache information may be stored in the shared cache if a processor is placed into a low power mode. One or more of the processors or cores in FIG. 3 may include or otherwise be associated with logic to monitor and/or control the scheduling or migration of processes/threads/tasks between each of the asymmetric cores of each processor.

FIG. 4 is a graph illustrating the performance and power characteristics associated with a processor when scaling voltage and frequency including techniques according to at least one embodiment of the invention. Reducing voltage is an efficient way of reducing power since the frequency scales linearly with the voltage, while the power scales as the voltage 3 (power=CV̂2F). Unfortunately, this efficient voltage scaling approach only works within a range of voltages; at some point, “Vmin”, the transistor switching frequency does not scale linearly with voltage. At this point (401), to further reduce power, the frequency is reduced without dropping the voltage. In this range, the power scales linearly with the frequency which is not nearly as attractive as when in the range where voltage scaling is possible. In one embodiment, power consumption of the system may be reduced below the minimum point 401 of a typical multi-core processor having symmetric processing elements by scheduling or migrating processes/threads/tasks from higher-performance/power cores to lower-performance/power cores if appropriate. In FIG. 4, the power/performance curve segment 405 indicates where the overall non-linear power/performance curve could be extended to enable more power savings, in one embodiment.

FIG. 5 illustrates a flow diagram of operations that may be used in conjunction with at least one embodiment of the invention. At operation 501, it is determined that a process/thread/task running on a main processor core of a multi-core processor does may be run on a lower power/performance core while maintaining an acceptable performance level. In one embodiment, the determination could be made by monitoring the activity level of the main core in response to running the thread/process/task and comparing it to a threshold value, corresponding to an acceptable performance metric of the lower power/performance core. In other embodiments, the determination could be made based on system power requirements, such as when the system is running on A/C power versus battery power. In yet other embodiments, a thread/process/task may be designated to require only a certain amount of processor performance, for example, by a programmer, the OS, etc. In other embodiments, other techniques for determining whether a task/thread/process could be transferred to a lower power/performance core, thereby reducing power consumption.

At operation 505, an event (e.g., yield, exception, etc.) occurs in the main core to cause state from the core to be saved and copied to a lower power/performance core. In one embodiment, a handler program is invoked in response to the event to cause the main core state to be transferred from the main core to a lower power/performance core. At operation 510, the transferred thread/process/task is restarted or resumed on the lower power/performance core. At operation 515, the main core may be placed in a lower power state (e.g., paused, halted, etc.) until 520 either the transferred process/task/thread requires above a threshold level of performance, in which case the thread/process/task may be transferred back to the main core 525 in a similar manner as it was transferred to the lower power/performance core, or another task/process/thread is scheduled for execution on the main core.

In one embodiment, the thread/process/task transferred from the main core to the lower power/performance core is first translated from the ISA of the main core to the ISA of the lower power/performance core, if the two have different architectures. For example, in one embodiment, the main core is an x86 architecture core and the lower power/performance core is an ARM architecture core, in which case instructions of the transferred thread/process/task may be translated (for example, by a software binary translation shell) from x86 instructions to ARM instructions. Because the thread/process/task being transferred is by definition one that does not require as much performance as to require it to be ran on the main core, a certain amount of latency may be tolerated in translating the process/task/thread from the x86 architecture to ARM architecture.

FIG. 6 illustrates a processing apparatus having a number of individual processing units between which processes/threads/tasks may be swapped under control of an activity level monitor, or thermal or power monitor, according to one embodiment. In the embodiment of FIG. 6, N processing units, processing unit 600-1, 600-2 through 600-N are coupled to a monitor or detection (generically referred to as “monitor”) logic 610. In one embodiment, the monitor 610 includes an activity, thermal and/or power monitoring unit that monitors the activity/performance, power consumption, and/or temperature of the processing units 600-1 through 600-N. In one embodiment, performance counters may be used to monitor the activity level of processing units 600-1 through 600-N. In one embodiment, the monitor 610 orchestrates process shifting between processing units in order to manage power consumption and/or particularly thermal concerns, while maintaining an acceptable level of performance.

In one embodiment, each processing unit provides a monitor value that typically reflects activity level, power consumption and/or temperature information to the monitor 610 via signals such as processor communication (PC) lines PC-1 through PC-N. The monitor value may take a variety of forms and may be a variety of different types of information. For example, the monitor value may simply be an analog or digital reading of the temperature of each processing unit. Alternatively, the monitor value may be a simple or complex activity factor that reflects the operational activity level of a particular processing unit. In some embodiments, power consumption information reflected by the monitor value may include a measured current level or other indication of how much power is being consumed by the processing unit. Additionally, some embodiments may convey power consumption information to the monitor 110 that is a composite of several of these or other types of known or otherwise available means of measuring or estimating power consumption. Accordingly, some power consumption metric which reflects one or more of these or other power consumption indicators may be derived. The transmitted monitor value may reflect a temperature or a power consumption metric, which itself may factor in a temperature. Serial, parallel, and/or various known or otherwise available protocols may be used to transmit this information to the power monitor.

In one embodiment, the monitor 610 receives the power consumption information from the various processing units and analyzes whether the power consumption or activity level of one processing unit is at a level to justify the overhead of re-allocating processes to different processing units. For example, the monitor may be triggered to rearrange processes when a particular processing unit falls below a threshold level of activity, or when power consumption is above an acceptable level. In one embodiment, the monitor 610 may develop a total power consumption metric to indicate the total power consumption, total activity level metric, or total thermal state of all processing units to effectuate the various power control strategies. In one embodiment, the monitor 610 may be a hardware component, a software component, routine, or module, or a combination of hardware and software that works either dependently or independently of the operating system.

In one embodiment, the monitor communicates to the processing units via thread or process swap control (SC) lines SC-1 through SC-N. The monitor is capable of moving and exchanging processes by sending commands via the SC lines. Thus, processes can be swapped between processing units, rotated between processing units, etc., in response to the particular chosen activity level, thermal, or power consumption metric being reached. Alternatively, or in addition to power consumption metric triggered process management, process rotation between processing units may be periodically performed to reduce the power consumption of the processor.

Many different types of processing devices could benefit from the use of such process re-allocation techniques. For example, the processing units 600-1 through 600-N may be general purpose processors (e.g., microprocessors) or may be microprocessor cores for a multiple core (on a single die) microprocessor. Alternatively, digital signal processors, graphics processors, network processors, or any type of special purpose processor that may be used in a system with multiple parallel units or cores may benefit from thermally (or power) motivated process shifting between processing units. The processing units or processors may be identical or have at least partial functional overlap. That is, each processing unit has some common set of instructions or commands such that there are at least some (if not all) processes that can be executed on more than one processing unit or processor. In other embodiments, the processing units may be asymmetrical, in as much as they have any or a combination of different performance capabilities, number of transistors, power consumption or thermal characterstics, clock frequencies, or ISA.

One or more aspects of at least one embodiment may be implemented by representative data stored on a machine-readable medium which represents various logic within the processor, which when read by a machine causes the machine to fabricate logic to perform the techniques described herein. Such representations, known as “IP cores” may be stored on a tangible, machine readable medium (“tape”) and supplied to various customers or manufacturing facilities to load into the fabrication machines that actually make the logic or processor.

Thus, a method and apparatus for directing micro-architectural memory region accesses has been described. It is to be understood that the above description is intended to be illustrative and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reading and understanding the above description. The scope of the invention should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. 

1. An apparatus comprising: at least two processing elements having different performance capabilities and power consumption properties; logic to determine whether instructions running on a first one of the at least two processing elements may be run on a second one of the at least two processing elements at or above a minimum level of performance while consuming no greater than a maximum level of power, wherein the instructions are to be transferred from the first to the second one of the at least two processing elements if the instructions may be run at or above the minimum level of performance while consuming no greater than the maximum level of power.
 2. The apparatus of claim 1, wherein the first of the at least two processing elements has a greater instruction-throughput capability than the second one of the at least two processing elements.
 3. The apparatus of claim 2, wherein state information corresponding to the instructions running on the first one of the at least two processing elements is to be transferred to the second one of the at least two processing elements if the instructions may be run at or above the minimum level of performance while consuming no greater than the maximum level of power.
 4. The apparatus of claim 1, wherein the first one of the at least two processing elements has a first instruction set architecture and the second one of the at least two processing elements has a second instruction set architecture.
 5. The apparatus of claim 4, wherein the instructions of the first instruction set architecture are to be translated into instructions of the second instruction set architecture.
 6. The apparatus of claim 1, wherein the first and second ones of the at least two processing elements have the same instruction set architecture.
 7. An apparatus comprising: a plurality of main processor cores and a plurality of lower-power processor cores, wherein the plurality of main cores has a higher performance capability than the plurality of lower-power processor cores, wherein a process being performed by one of the plurality of main processor cores is to be performed by one of the plurality of lower-power processor cores in response to a change in power consumption condition, wherein the one of the plurality of main processor cores is to be placed in a low-power state.
 8. The apparatus of claim 7, wherein the plurality of main processor cores have a greater instruction throughput capability than the plurality of lower-power cores.
 9. The apparatus of claim 8, wherein state information corresponding to instructions running on the plurality of main processor cores is to be transferred to the plurality of lower-power processor cores if the instructions may be run at or above the minimum level of performance while consuming no greater than the maximum level of power.
 10. The apparatus of claim 7, wherein the plurality of main processor cores have a first instruction set architecture and the plurality of lower-power processor cores have a second instruction set architecture.
 11. The apparatus of claim 10, wherein the instructions of the first instruction set architecture are to be translated into instructions of the second instruction set architecture.
 12. The apparatus of claim 7, wherein the plurality of main processor cores and the plurality of lower-power processor cores have the same instruction set architecture.
 13. An apparatus comprising: at least one first processor core having a first instruction set architecture (ISA) and a first performance capability; at least one second processor core having a second ISA and a second performance capability, wherein the second performance capability is less than the first performance capability; at least a first logic to cause instructions corresponding to the first ISA to be transferred to the at least second processor core and translated from the first ISA to the second ISA.
 14. The apparatus of claim 13, wherein state information corresponding to instructions running on the at least one first processor core is to be transferred to the at least second cores if the instructions may be run at or above the minimum level of performance while consuming no greater than the maximum level of power.
 15. An apparatus comprising: activity level detection logic to determine whether an activity level of a first processor core performing a first task is below a first threshold, wherein if the first task is below the first threshold, the first task is to be transferred to a second processor core, wherein the second processor core is to consume less power than the first processor core at the same activity level.
 16. The apparatus of claim 15 having at least one first processor core having a first instruction a first performance capability; at least one second processor core having a second performance capability, wherein the second performance capability is less than the first performance capability.
 17. The apparatus of claim 16, wherein the at least on first processor core has a first instruction set architecture and the at least one second processor cores has a second instruction set architecture.
 18. The apparatus of claim 17, wherein the instructions of the first instruction set architecture are to be translated into instructions of the second instruction set architecture.
 19. The apparatus of claim 15, wherein the plurality of main processor cores and the plurality of lower-power processor cores have the same instruction set architecture.
 20. A mobile computer system comprising: at least one first processor having a first minimum power consumption capability and a first instruction set architecture to run a first operating system; at least one second processor having a second minimum power consumption capability having a second instruction set architecture to run a second operating system having a different kernel than the first operation system, wherein the first minimum power consumption capability is greater than the second minimum power consumption capability.
 21. The mobile computer system of claim 20, wherein the at least one first processor has a different maximum performance capability that is different from that of the at least one second processor.
 22. The mobile computer system of claim 20, wherein first instruction set architecture is an x86 instruction set architecture and the second instruction set architecture is an advanced RISC machine (ARM) instruction set architecture.
 23. The mobile computer system of claim 20, wherein the first operating system is a Windows® operating system and the second operating system is a Linux® operating system.
 24. The mobile computer system of claim 20, wherein the at least one first processor is to perform applications requiring more performance than applications to be ran on the at least one second processor. 